Generic video analysis systems are used to determine the presence of people, vehicles or other specific objects (targets) in a certain area of space from images captured by an image acquisition device, preferably a fixed camera observing that area of space.
If the input image of this video analysis system is either an IR (infrared) or a thermal spectrum image, one of the main problems that the system faces is the lack of contrast between the background and foreground of the scene, thus making target detection more difficult. This is particularly the case with the thermal spectrum at warmer times of the year when ground temperatures can reach 37° C. and the detection of human targets is hindered due to lack of image contrast. This effect is further exacerbated in remote areas of the image where objects are smaller. This problem not only affects automatic video surveillance or analysis systems but also operator-verified systems.
Manufacturers of infrared spectrum cameras have attempted to alleviate the problem by adding software tools to their sensors to reduce contrast issues and improve image quality. The algorithms developed for this purpose enable adjustment of brightness or gain, basic histogram equalization or complex plateau-type histogram equalization or detail enhancement filters. For their part, manufacturers of video analysis systems, aware of these difficulties, have incorporated image and/or contrast enhancement modules to mitigate the problem.
The main drawback of the above-commented algorithms, however, is that enhancement is only based on information present in the image. In other words, they treat all pixels equally without making any assumptions about their nature (e.g. distance from which the temperature is projected or size of the person in the location).
However, it should be noted that the scientific literature includes documents that describe image enhancement methods based on scene and/or depth information, in which visible spectrum cameras are mainly used. The most significant documents are the following:                On the one hand, there are documents such as a patent application with publication number US2010/0304854 entitled “Image contrast enhancement in depth sensor” and a paper by Hachicha et al. presented at the 20th European Signal Processing Conference (EUSIPCO 2012), entitled “Combining depth information and the local edge detection for stereo image enhancement” in which the goal is not to enhance the quality of the image using a depth reference but to improve that depth estimation based on the image analyzed. In short, the above-mentioned process is performed in reverse.                    The first of the documents cited, as an example, describes a system whose calculation of image depth is based on emitting structured light and analyzing its projection on the scene. The document seeks to enhance the quality of the projected structured light image to improve depth estimation.            In the second cited document, as an example, there is a stereo pair (visible image+depth map) and the method seeks to segment the objects in the image, enhance contrast and apply this information to improve the depth map.                        On the other hand, there are documents such as the paper presented by Hu et al. at ICIP 2013, entitled “Kinect depth map based enhancement for low light surveillance image” and a patent application with publication number CN103400351A entitled “Low illumination image enhancing method and system based on KINECT depth graph”, in which information from external devices (e.g. Kinect cameras) are applied to visible spectrum cameras. These devices provide us with a depth map of the scene associated with the image of the scene being observed with the visible spectrum camera in order to enable enhancement operations on this camera to be performed. Specifically, in the first case, a noise filter and histogram adjustment are performed, in which the depth image is incorporated into the overall image equalization.        A large number of references also exist, such as a document by Srinivasa et al. in IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 6, June 2003, entitled “Contrast restoration of weather degraded images” and a document by Munira at al. in the International Journal of Scientific and Research Publications, Volume 3, Issue 6, June 2013, entitled “Single image fog removal using depth estimation based on blur estimation” whose methods aim to enhance the quality of images that are affected by weather conditions (e.g. fog or pollution) by estimating the depth of the scene based precisely on these effects. Specifically, the more blurred an area of the image, the further away the camera. Once the depth estimation is obtained, it is used to cancel the effects on the image of the atmospheric conditions, thereby enhancing its quality.        Another group of documents should also be noted, such as a patent with publication number U.S. Pat. No. 7,706,576, entitled “Dynamic video equalization of images using face-tracing”, in which objects of interest are detected to improve their quality. In the document cited as an example, t is assumed that the face of a person is visible in the scene, this face is detected and an algorithm is applied to enhance the image in the region occupied by the face. In reality, scene information is being applied in order to enhance at least one area of the image.        
In terms of scientific documents relating to methods for improving IR thermal images, the most significant, among the few that exist, are a patent application with publication number EP2226762, entitled “Equalization and processing of IR images” and a PCT patent application with publication number WO2013126001, entitled “Image processing method for detail enhancement and noise reduction”, which use local or overall information from the image itself, but never any depth or scene information. The main drawback of these methods is that the images still contain areas of insufficient contrast to detect objects.
The present invention is an image enhancement method for video analysis or automatic surveillance systems whose input image is either an IR spectrum image or a thermal spectrum image, in which depth or scene information is used to provide the resulting image with sufficient contrast to determine the presence of a particular object.